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Amendments to the Claims : 

(This listing of claims replaces all prior versions and listings of claims.) 

1 . (previously presented) A machine-based method for use in processing a digital 
image that includes first and second regions, the method comprising: 

estimating an intrinsic color of a given pixel located in an area of interest that is adjacent 
to at least one of the first and second regions, the estimating comprises extrapolating from colors 
of multiple pixels in one of the first and second regions and multiple pixels in the other of the 
two regions, and 

storing the intrinsic color of the pixel for later use. 

2. (previously presented) The method of claim 1 in which the given pixel has an 
original color that relates to the original colors of pixels in both the first and second regions, and 
the estimated intrinsic color of the given pixel relates to original colors in only one or the other 
of the first and second regions. 

3. (previously presented) The method of claim 1 in which the area of interest 
includes one of the first and second regions. 

4. (previously presented) The method of claim 1 in which the area of interest is 
adjacent to both of the first and second regions. 

5. (previously presented) The method of claim 1 in which the first region comprises 
a foreground object and the second region comprises a background. 

6. (previously presented) The method of claim 1 in which the first and second 
regions may have any arbitrary degree of color variation in the visible spectrum over a spatial 
scale that is on the same order of magnitude or smaller than the minimum span of the area of 
interest. 

7. (previously presented) The method of claim 1 in which the estimating comprises 
analyzing both the color and spatial proximity of pixels in the first and second regions. 

8. (previously presented) The method of claim 1 in which the estimating comprises 
extrapolating from the closest pixels in the first and second regions. 
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9. (previously presented) The method of claim 1 in which the estimating comprises 
flowing colors into the area of interest from one or both of the first and second regions. 

10. (previously presented) The method of claim 9 in which the flowing of colors 
comprises averaging of color values for each of a set of pixels in the first region and a set of 
pixels in the second region. 

1 1 . (previously presented) The method of claim 1 in which the digital image 
comprises layers of pixel information and the estimating is based on pixel information in only 
one of the layers. 

12. (previously presented) The method of claim 1 in which the digital image 
comprises layers of pixel information and the estimating is based on pixel information in a 
composition of all the layers. 

13. (previously presented) The method of claim 1 further comprising determining an 
opacity value for the given pixel, indicative of the extent to which the intrinsic color of the given 
pixel relates to original colors in the first and second regions, based on a result of the estimating 
of the intrinsic color. 

14. (previously presented) The method of claim 1 3 in which the given pixel includes 
original opacity information, and the opacity value is also based on the original opacity 
information. 

15. (previously presented) The method of claim 13 further comprising 
determining opacity values for other pixels that have intrinsic colors that relate to original colors 
in the first and second regions. 

16. (previously presented) The method of claim 13 in which the opacity 
determination comprises use of a neural network trained on the image original colors and 
estimated intrinsic colors. 

1 7. (previously presented) The method of claim 1 3 further comprising using the 
opacity value to composite one of the first and second regions with another digital image. 

18. (previously presented) The method of claim 1 in which the estimating also 
includes extrapolating estimates of intrinsic colors of the first and second regions using searches 
in color space and image coordinate space. 
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19. (previously presented) The method of claim 1 in which the estimating assumes a 
linear blending model. 

20. (previously presented) The method of claim 1 in which the estimating includes 
flowing colors from edges of the area of interest to fill the area of interest with estimates of the 
colors of the first and second regions. 

21. (previously presented) The method of claim 1 further comprising 

extracting from the digital image the intrinsic colors of the given pixel and of other pixels 
that have intrinsic colors that relate to original colors in the first region or second region. 

22. (previously presented) The method of claim 21 further comprising using the 
extracted intrinsic colors to composite the first region or the second region with another digital 
image. 

23. (previously presented) The method of claim 1 further comprising 

receiving from an interactive user interface information that identifies the area of interest. 

24. (previously presented) The method of claim 1 in which estimating the intrinsic 
color comprises 

determining two color sample sets for the given pixel, each of the color sample sets being 
associated with one of the first and second regions, and 

estimating the intrinsic color based on the two color sample sets. 

25. (previously presented) The method of claim 24 in which estimating the intrinsic 
color comprises comparing the original color of the given pixel with colors in the color sample 
sets. 

26. (previously presented) The method of claim 24 further comprising 
determining an opacity for the given pixel indicative of the extent to which the intrinsic 

color of the given pixel relates to original colors in both of the first and second regions, where 
the determination of opacity includes comparing the original color of the given pixel with colors 
in the color sample sets. 

27. (previously presented) The method of claim 26 in which the given pixel includes 
original opacity information and the determination of opacity is also based on the original 
opacity information. 
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28. (previously presented) The method of claim 24 in which the color sample sets are 
derived from colors of pixels in the first and second regions. 

29. (previously presented) The method of claim 24 in which a single color is selected 
from each of the color sample sets based on an error minimization technique. 

30. (previously presented) The method of claim 1 in which the intrinsic colors of all 
of the pixels in the area of interest are determined automatically. 

31-42. (cancelled). 

43. (previously presented) A method for use in processing a digital image, 
comprising 

receiving a mask associated with an area of interest in the digital image, the mask 
including values representing opacities of pixels in the region of interest with respect to an 
adjacent region of interest, and 

based on the mask, estimating intrinsic colors for the pixels. 

44. (previously presented) A machine-based method for use in extracting a 
foreground region from a background region of an image, comprising 

enabling a user to control an original extraction by manipulating a brush on a display of 
the image, 

enabling a user to control a touch up extraction following the original extraction, and 
considering a pixel identified for touch up extraction only if the pixel was of uncertain 
color in the original extraction. 

45. (previously presented) The method of claim 44 in which an intrinsic color is 
determined for each of the pixels that were of uncertain color based on a forced foreground or 
background color. 

46. (previously presented) The method of claim 44 in which the forced color is 
selected by the user. 

47. (previously presented) The method of claim 44 in which the forced color is 
determined automatically from the original colors within the foreground region. 

48. (previously presented) A method for use in determining, for each pixel in an area 
of interest in a digital image, the nearest pixel in a first region of the image that is adjacent to the 
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area of interest and the nearest pixel in a second region of the image that is adjacent to the area of 

interest, the method comprising 

defining a processing area that is smaller than the image, 

defining a pixel window that is smaller than the defined processing area, 

scanning the processing area to a succession of overlapping positions that together span 

the image 

at each overlapping position of the processing area, scanning the pixel window across the 
processing area, and 

at each position of scanning of the pixel window, updating stored information for pixels 
in the window, the stored information relating to nearest pixels in the first and second regions. 

49. (previously presented) The method of claim 48 in which the processing area 
comprises a rectangle twice as long is high, and in each of the succession of positions the 
processing area is offset from the prior position by half the length of the rectangle. 

50. (previously presented) The method of claim 48 in which the pixel window 
comprises a square. 

5 1 . (previously presented) The method of claim 48 in which the scanning of the 
processing area and the scanning of the pixel window occur in both forward and backward 
passes that span the image. 

52. (previously presented). The method of claim 48 further comprising 
extrapolating colors from the nearest pixels. 

53. (previously presented) The method of claim 48 in which the first region 
comprises a foreground object, the second region comprises a background, and at least some 
pixels in the area of interest have uncertain color. 

54. (previously presented) A machine-based method for a user to extract an object 
from a background in an image, comprising 

displaying the image, 

selecting a painting tool and adjusting its characteristics, 
using the painting tool to paint a swath around the object, 
the swath including pixels whose membership in the object or the background are 
uncertain and including pixels that with certainty belong to the object and to the background 



Applicant : Gregg D. Wilensky, et al. Attorney's Docket No.: 07844-329002 / P303C1 

Serial No. : 

Filed : September 5, 2003 
Page : 8 of 10 

indicating at least one pixel that is known to belong to the object or the background, 
invoking a program to perform the extraction, 
observing whether the quality of the extraction, and 

depending on the observation, using a painting tool to control a touch-up extraction. 

55. (previously presented) A medium bearing a computer program capable of 
controlling a computer to process a digital image that includes first and second regions by: 

estimating an intrinsic color of a given pixel located in an area of interest that is adjacent 
to at least one of the first and second regions, the estimating comprises extrapolating from colors 
of multiple pixels in one of the first and second regions and multiple pixels in the other of the 
two regions, and 

storing the intrinsic color of the pixel for later use. 

56. (cancelled). 

57. (previously presented) A medium bearing a computer program capable of 
controlling a computer to extract a foreground region from a background region of an image by 

enabling a user to control an original extraction by manipulating a brush on a display of 
the image, 

enabling a user to control a touch up extraction following the original extraction, and 
considering a pixel identified for touch up extraction only if the pixel was of uncertain 
color in the original extraction. 

58. (previously presented) A system for use in processing a digital image that 
includes first and second regions, the system comprising: 

means for estimating an intrinsic color of a given pixel located in an area of interest that 
is adjacent to at least one of the first and second regions, the estimating comprises extrapolating 
from colors of multiple pixels in one of the first and second regions and multiple pixels in the 
other of the two regions, and 

means for storing the intrinsic color of the pixel for later use. 

59. (cancelled). 

60. (previously presented) A system for use in extracting a foreground region from a 
background region of an image, comprising 
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means for enabling a user to control an original extraction by manipulating a brush on a 

display of the image, 

means for enabling a user to control a touch up extraction following the original 

extraction, and 

means for considering a pixel identified for touch up extraction only if the pixel was of 
uncertain color in the original extraction. 



